<html>
<head><meta charset="utf-8"><title>Analyzing a perf regression · t-compiler/performance · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/index.html">t-compiler/performance</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html">Analyzing a perf regression</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="244079676"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244079676" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244079676">(Jun 27 2021 at 18:50)</a>:</h4>
<p>Okay, so, I have a problem</p>



<a name="244079686"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244079686" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244079686">(Jun 27 2021 at 18:50)</a>:</h4>
<p>I have a PR: <a href="https://github.com/rust-lang/rust/issues/85499">#85499</a></p>



<a name="244079714"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244079714" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244079714">(Jun 27 2021 at 18:51)</a>:</h4>
<p>It currently has a perf regression in multiple benchmarks, which the largest being inflate-check/debug and keccake-check/debug</p>



<a name="244079773"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244079773" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244079773">(Jun 27 2021 at 18:52)</a>:</h4>
<p>Now, prior to <a href="https://github.com/rust-lang/rust/pull/85499/commits/8cb81f5d9b4e54064e2760f449c51f456b320d68">this</a> commit, it was showing about a 10% regression in instruction counts</p>



<a name="244079786"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244079786" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244079786">(Jun 27 2021 at 18:52)</a>:</h4>
<p>However, <span class="user-mention" data-user-id="330154">@The 8472</span> has been helping me check this locally (their local), and has now been able to see really any regression at all</p>



<a name="244079819"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244079819" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244079819">(Jun 27 2021 at 18:53)</a>:</h4>
<div class="message_inline_image"><a href="https://zulip-uploads.s3.amazonaws.com/4715/WIOn4BtaarNTzvc6SQP7CBcI/perf.png?AWSAccessKeyId=AKIAIEVMBCAT2WD3M5KQ&amp;Signature=1H3CLExzkHG8D4fYrhJRa2y52HU%3D&amp;Expires=1624820058"><img src="https://uploads.zulipusercontent.net/f8e2fc38960e31bd98f73d1165ce825685667ff9/68747470733a2f2f7a756c69702d75706c6f6164732e73332e616d617a6f6e6177732e636f6d2f343731352f57494f6e3442746161724e547a76633653515037434263492f706572662e706e673f4157534163636573734b657949643d414b49414945564d42434154325744334d354b51265369676e61747572653d314833434c45787a6b4847384434665972684a526132793532485525334426457870697265733d31363234383230303538"></a></div>



<a name="244079843"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244079843" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244079843">(Jun 27 2021 at 18:54)</a>:</h4>
<p><em>After</em> that commit (and a rebase, but I don't think that's actually relevant), there's actually a slight perf <em>improvement</em></p>



<a name="244079876"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244079876" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244079876">(Jun 27 2021 at 18:54)</a>:</h4>
<div class="message_inline_image"><a href="https://zulip-uploads.s3.amazonaws.com/4715/ik6NlCqPiQbgqrjfD5CQWuq0/perf.png?AWSAccessKeyId=AKIAIEVMBCAT2WD3M5KQ&amp;Signature=P9mEqG%2FlV2SSGZPTj48ml3Z%2FFTQ%3D&amp;Expires=1624820109"><img src="https://uploads.zulipusercontent.net/a9f008e4fb860093d7c60071eadd67e5e259a8e8/68747470733a2f2f7a756c69702d75706c6f6164732e73332e616d617a6f6e6177732e636f6d2f343731352f696b364e6c4371506951626771726a6644354351577571302f706572662e706e673f4157534163636573734b657949643d414b49414945564d42434154325744334d354b51265369676e61747572653d50396d4571472532466c56325353475a50546a34386d6c335a25324646545125334426457870697265733d31363234383230313039"></a></div>



<a name="244079924"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244079924" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244079924">(Jun 27 2021 at 18:56)</a>:</h4>
<p>Running through <code>perf</code> for the branch before the rebase, compared to that git base also didn't show really any meaningful differences</p>



<a name="244079971"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244079971" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244079971">(Jun 27 2021 at 18:56)</a>:</h4>
<p>I've also analyzed the logs from <code>RUSTC_LOG</code> and nothing is really popping out (though, they're a bit hard to analyze since these end up giving some large logs)</p>



<a name="244079983"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244079983" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244079983">(Jun 27 2021 at 18:57)</a>:</h4>
<p>So, the question: what might be giving such a large regression on perf.rlo and what can I/we do to help reproduce that locally</p>



<a name="244080049"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244080049" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244080049">(Jun 27 2021 at 18:58)</a>:</h4>
<p><span class="user-mention silent" data-user-id="232957">Jack Huey</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/Analyzing.20a.20perf.20regression/near/244079843">said</a>:</p>
<blockquote>
<p><em>After</em> that commit (and a rebase, but I don't think that's actually relevant), there's actually a slight perf <em>improvement</em></p>
</blockquote>
<p>Also worth mentioning that  this was also local. A perf run with that commit, but not a rebase, seems to have improved a few percent: <a href="https://perf.rust-lang.org/compare.html?start=a5b7511a6ccf241ba2ce6ad0b04e79f1e3d85686&amp;end=df68c4714df1c1a04afeb0677193895d9c80fa9a">https://perf.rust-lang.org/compare.html?start=a5b7511a6ccf241ba2ce6ad0b04e79f1e3d85686&amp;end=df68c4714df1c1a04afeb0677193895d9c80fa9a</a></p>



<a name="244080143"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244080143" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244080143">(Jun 27 2021 at 19:01)</a>:</h4>
<p><span class="user-mention silent" data-user-id="232957">Jack Huey</span> <a href="#narrow/stream/247081-t-compiler.2Fperformance/topic/Analyzing.20a.20perf.20regression/near/244079983">said</a>:</p>
<blockquote>
<p>So, the question: what might be giving such a large regression on perf.rlo and what can I/we do to help reproduce that locally</p>
</blockquote>
<p>And to followup on this: how reasonable is it to just...land this PR with perf as-is. It's already a huge win  for deeply-nested-async. And short of some extra caching, some inlining attributes, or something similar somewhere, I don't really know what I can reasonably do in this PR <em>anyways</em></p>



<a name="244080711"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244080711" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244080711">(Jun 27 2021 at 19:17)</a>:</h4>
<p>If out of 3 perf runs 2 give reasonable results that you might expect, I think its fine to ignore the one-off. Trying and failing to reproduce locally I think is sufficient indication of attempting to figure out the problem.</p>



<a name="244080722"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244080722" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244080722">(Jun 27 2021 at 19:18)</a>:</h4>
<p>It would be a different problem if every single perf run was bad, but not reproducible locally.</p>



<a name="244081933"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244081933" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244081933">(Jun 27 2021 at 19:50)</a>:</h4>
<p>Well, the perf runs on perf.rlo are consistently bad</p>



<a name="244081939"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244081939" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244081939">(Jun 27 2021 at 19:51)</a>:</h4>
<p>whereas locally, the are neutral to <em>positive</em></p>



<a name="244083060"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244083060" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> lqd <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244083060">(Jun 27 2021 at 20:21)</a>:</h4>
<p>about the discrepancy: are these runs all measuring the same thing ? (the linked commit makes it look like debug assertions and logging are turned on, while changing logging like this wouldn’t be visible on perf.rlo IIRC ?) what do the images above show btw ? they seem to be broken links for me</p>



<a name="244085658"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244085658" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244085658">(Jun 27 2021 at 21:24)</a>:</h4>
<p><span class="user-mention" data-user-id="232957">@Jack Huey</span> can you reproduce regressions locally using the artifacts from perf? I think landing the regression may be reasonable -- haven't looked too much into things, would need to do more to really comment there. But if perf is showing drastically different results to what you're seeing locally, it's possible that we can find wins by adjusting configurations or whatever else to get you nicer results locally.</p>



<a name="244088320"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088320" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088320">(Jun 27 2021 at 22:28)</a>:</h4>
<p><span class="user-mention" data-user-id="116113">@lqd</span> they should all be measuring the same thing. The important bit of the linked commit is the online attributes. The images show first the neutral local results and second the improved local results</p>



<a name="244088378"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088378" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088378">(Jun 27 2021 at 22:30)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> where can I get the artifacts? Perf is showing bad results, but local isn't. Not sure how to be able to actually figure out the regression on perf.rlo</p>



<a name="244088388"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088388" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088388">(Jun 27 2021 at 22:30)</a>:</h4>
<p>rustup toolchain install master</p>



<a name="244088398"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088398" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088398">(Jun 27 2021 at 22:31)</a>:</h4>
<p>(it's a crate on <a href="http://crates.io">crates.io</a>, can install try builds)</p>



<a name="244088459"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088459" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088459">(Jun 27 2021 at 22:32)</a>:</h4>
<p>Is this documented anywhere?</p>



<a name="244088470"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088470" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088470">(Jun 27 2021 at 22:33)</a>:</h4>
<p>Hm not sure maybe</p>



<a name="244088538"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088538" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088538">(Jun 27 2021 at 22:34)</a>:</h4>
<p><a href="https://crates.io/crates/rustup-toolchain-install-master/">https://crates.io/crates/rustup-toolchain-install-master/</a></p>



<a name="244088552"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088552" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088552">(Jun 27 2021 at 22:35)</a>:</h4>
<p><span class="user-mention" data-user-id="330154">@The 8472</span> do you think you could run perf on the try build?</p>



<a name="244088602"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088602" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088602">(Jun 27 2021 at 22:36)</a>:</h4>
<p><a href="https://github.com/rust-lang/rust/pull/85499#issuecomment-869115256">https://github.com/rust-lang/rust/pull/85499#issuecomment-869115256</a></p>



<a name="244088608"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088608" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088608">(Jun 27 2021 at 22:36)</a>:</h4>
<p>I guess the base is also built too, right?</p>



<a name="244088618"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088618" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088618">(Jun 27 2021 at 22:37)</a>:</h4>
<p>yeah, will take a bit</p>



<a name="244088628"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088628" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088628">(Jun 27 2021 at 22:37)</a>:</h4>
<p><a href="https://github.com/rust-lang/rust/commit/49ba9361d87f5d58d54fdd164ef2128cd50ed4b4">49ba9361d87f5d58d54fdd164ef2128cd50ed4b4</a> is the base I think</p>



<a name="244088759"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088759" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088759">(Jun 27 2021 at 22:41)</a>:</h4>
<p>you can see the actual commits that perf is comparing in the perf results</p>



<a name="244088877"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088877" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088877">(Jun 27 2021 at 22:44)</a>:</h4>
<p><a href="/user_uploads/4715/8342iP9REmuX7UANgfNKUwVa/perf.png">perf.png</a> </p>
<div class="message_inline_image"><a href="/user_uploads/4715/8342iP9REmuX7UANgfNKUwVa/perf.png" title="perf.png"><img src="/user_uploads/4715/8342iP9REmuX7UANgfNKUwVa/perf.png"></a></div><p>Well, those results don't match those from my local builds.</p>



<a name="244088953"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088953" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088953">(Jun 27 2021 at 22:47)</a>:</h4>
<p>But they're pretty close to what perf rlo shows. So I wonder why I'm seeing a much smaller difference with local builds.</p>



<a name="244088954"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244088954" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244088954">(Jun 27 2021 at 22:47)</a>:</h4>
<p>Note that CI builds are PGO and have, e.g., codegen-units-std = 1 along with things like debug! off... so there could be quite a bit of difference. But the builds tested by perf <em>are</em> what we ship.</p>



<a name="244089179"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244089179" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244089179">(Jun 27 2021 at 22:53)</a>:</h4>
<p>Yesterday I was testing different commits, before a rebase and that was about 0.4% regression with 16CGUs and no rustflags. With 1CGU and RUSTFLAGS=-Ctarget-cpu=native it was a 0.1% regression. Neither matched what perf rlo was showing.<br>
debug=false, debuginfo=1 or 2, don't remember which one<br>
no PGO</p>



<a name="244089234"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244089234" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244089234">(Jun 27 2021 at 22:54)</a>:</h4>
<p>I only ran keccak and inflate since those were the top regressions.</p>



<a name="244089695"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244089695" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244089695">(Jun 27 2021 at 23:08)</a>:</h4>
<p>You can try to locally build the same way perf does with the docker container (src/ci/docker/run.sh dist-x86_64-linux)</p>



<a name="244089705"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244089705" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244089705">(Jun 27 2021 at 23:08)</a>:</h4>
<p>Certainly perf doesn't use =native codegen, so not really sure why that's a point of comparison...</p>



<a name="244089723"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244089723" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244089723">(Jun 27 2021 at 23:09)</a>:</h4>
<p><span class="user-mention" data-user-id="330154">@The 8472</span> if you do perf record then perf diff on those two builds, what does that show?</p>



<a name="244089725"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244089725" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244089725">(Jun 27 2021 at 23:09)</a>:</h4>
<p>Might actually get something here</p>



<a name="244089778"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244089778" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244089778">(Jun 27 2021 at 23:11)</a>:</h4>
<blockquote>
<p>Certainly perf doesn't use =native codegen, so not really sure why that's a point of comparison...</p>
</blockquote>
<p>that's just my default environment, I only remembered that I had it set at some point and then turned it off. unintended point of data</p>



<a name="244090336"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244090336" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> The 8472 <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244090336">(Jun 27 2021 at 23:26)</a>:</h4>
<p>diff output for the try builds <a href="https://gist.github.com/the8472/b92600421a296a26828030f98d210ee8">https://gist.github.com/the8472/b92600421a296a26828030f98d210ee8</a></p>



<a name="244091025"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244091025" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244091025">(Jun 27 2021 at 23:47)</a>:</h4>
<p>Interesting</p>



<a name="244091248"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244091248" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244091248">(Jun 27 2021 at 23:52)</a>:</h4>
<p>I wonder if PGO is optimising process_obligations incorrectly</p>



<a name="244091444"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244091444" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244091444">(Jun 27 2021 at 23:58)</a>:</h4>
<p>Actually, looks like maybe <code>apply_rewrites</code> got inlined. Before it was ~3% of time. Now process_obligations got 5% longer</p>



<a name="244091447"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244091447" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244091447">(Jun 27 2021 at 23:59)</a>:</h4>
<p>Then also that HashMap::retain</p>



<a name="244091525"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244091525" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244091525">(Jun 28 2021 at 00:01)</a>:</h4>
<p>Oh hmm. <code>apply_rewrites</code> is called from <code>compress</code>, which isn't inlined</p>



<a name="244092770"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244092770" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244092770">(Jun 28 2021 at 00:38)</a>:</h4>
<p>You can try adding some of the regressed benchmarks to the pgo script (src/ci/pgo.sh I think? Something like that), maybe it'll help</p>



<a name="244092773"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244092773" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244092773">(Jun 28 2021 at 00:38)</a>:</h4>
<p>But if you do end up seeing wins with that it shouldn't land with any other changes ultimately</p>



<a name="244094595"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244094595" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244094595">(Jun 28 2021 at 01:32)</a>:</h4>
<p>Well, <code>inflate</code> isn't edition 2018: <a href="https://github.com/rust-lang/rust/pull/85499#issuecomment-869268462">https://github.com/rust-lang/rust/pull/85499#issuecomment-869268462</a></p>



<a name="244189863"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244189863" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244189863">(Jun 28 2021 at 18:55)</a>:</h4>
<p>Okay, so I added <code>inline(never)</code> to <code>apply_rewrites</code> and added <code>inflate</code> to the pgo script. It looks like the former had about a 0.2% improvment (within noise) (<a href="https://perf.rust-lang.org/compare.html?start=a5b7511a6ccf241ba2ce6ad0b04e79f1e3d85686&amp;end=df68c4714df1c1a04afeb0677193895d9c80fa9a">without</a> vs <a href="https://perf.rust-lang.org/compare.html?start=17ea490310ba7c836c93fe1b7002555b3bea5eb1&amp;end=735792470d44bbc1c303758b6f543b54e8ef8964">with</a>) whereas the latter did have a 1-2% improvement (<a href="https://perf.rust-lang.org/compare.html?start=17ea490310ba7c836c93fe1b7002555b3bea5eb1&amp;end=735792470d44bbc1c303758b6f543b54e8ef8964">without</a> vs <a href="https://perf.rust-lang.org/compare.html?start=3e9d7ecf784e5ecaf7437d04be3992ad23fa7cb6&amp;end=d6a65aea9952f5f55308e7f4754b06ca9cfd1532">with</a>)</p>



<a name="244190177"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244190177" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244190177">(Jun 28 2021 at 18:57)</a>:</h4>
<p>That leaves the the <a href="https://github.com/rust-lang/rust/pull/85499#issuecomment-869380399">current</a> perf at about 30% improvement for deeply-nested-async and 1-4% regression on a dozen or so benchmarks.</p>



<a name="244190383"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244190383" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244190383">(Jun 28 2021 at 18:59)</a>:</h4>
<p>As <span class="user-mention" data-user-id="116122">@simulacrum</span> said, I can make a separate PR for the pgo script edit. It'll just make the numbers on the current PR look worse again.</p>



<a name="244190461"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244190461" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244190461">(Jun 28 2021 at 19:00)</a>:</h4>
<p>But I think the benefit of landing this PR heavily outweighs the couple percent regression.</p>



<a name="244191237"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244191237" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244191237">(Jun 28 2021 at 19:07)</a>:</h4>
<p><a href="https://github.com/rust-lang/rust/issues/86697">#86697</a> okay let's see</p>



<a name="244194224"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244194224" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244194224">(Jun 28 2021 at 19:35)</a>:</h4>
<p><span class="user-mention" data-user-id="232957">@Jack Huey</span> Yes, I think we can likely proceed -- you did want to do a crater run as well it looks like?</p>



<a name="244194320"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244194320" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244194320">(Jun 28 2021 at 19:36)</a>:</h4>
<p>Hmm. Likely a good idea...</p>



<a name="244194325"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244194325" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244194325">(Jun 28 2021 at 19:36)</a>:</h4>
<p>how's the queue...</p>



<a name="244194331"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/247081-t-compiler/performance/topic/Analyzing%20a%20perf%20regression/near/244194331" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jack Huey <a href="https://rust-lang.github.io/zulip_archive/stream/247081-t-compiler/performance/topic/Analyzing.20a.20perf.20regression.html#244194331">(Jun 28 2021 at 19:36)</a>:</h4>
<p>oh, pretty empty</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>